1
00:00:00,790 --> 00:00:07,320
[Music]

2
00:00:12,850 --> 00:00:09,240
[Applause]

3
00:00:16,089 --> 00:00:12,860
I'm going to talk about sampling as well

4
00:00:17,800 --> 00:00:16,099
but using robotic manipulators and more

5
00:00:18,760 --> 00:00:17,810
likely how are we going from

6
00:00:22,659 --> 00:00:18,770
teleoperation

7
00:00:24,609 --> 00:00:22,669
to autonomy so first of all was the

8
00:00:27,790 --> 00:00:24,619
comparison between the deep sea and the

9
00:00:30,549 --> 00:00:27,800
deep space is that they are quite alike

10
00:00:32,860 --> 00:00:30,559
in order to use the Earth's ocean in

11
00:00:34,900 --> 00:00:32,870
order to study the space and nASA has

12
00:00:37,450 --> 00:00:34,910
been done that and in the past years

13
00:00:39,910 --> 00:00:37,460
they have funded a few projects and one

14
00:00:42,279 --> 00:00:39,920
of them is the project that we work on

15
00:00:45,720 --> 00:00:42,289
and it was a pistol project and the idea

16
00:00:49,000 --> 00:00:45,730
was to take these robotic manipulators

17
00:00:51,639 --> 00:00:49,010
underwater to collect samples in

18
00:00:54,840 --> 00:00:51,649
underwater volcanoes but then working at

19
00:00:57,939 --> 00:00:54,850
very high depth so we wanted to collect

20
00:01:01,599 --> 00:00:57,949
sediments of the soil in these volcanoes

21
00:01:04,420 --> 00:01:01,609
in order to study the ocean floor what

22
00:01:06,219 --> 00:01:04,430
kind of composites are in there and how

23
00:01:10,480 --> 00:01:06,229
this affect their life and this can be

24
00:01:14,710 --> 00:01:10,490
translated to any other outer planet to

25
00:01:17,410 --> 00:01:14,720
study so in how can we do that we saw

26
00:01:20,310 --> 00:01:17,420
earlier the Landers it's an option but

27
00:01:22,990 --> 00:01:20,320
we want to have more autonomy and like

28
00:01:26,260 --> 00:01:23,000
enlarge the space where we can work at

29
00:01:28,990 --> 00:01:26,270
so a robotic manipulator it's a good

30
00:01:31,840 --> 00:01:29,000
idea to to use that and this kind of

31
00:01:35,680 --> 00:01:31,850
robots that are currently used for the

32
00:01:38,530 --> 00:01:35,690
this applications are large pieces of

33
00:01:41,530 --> 00:01:38,540
equipment that are mostly hydraulic and

34
00:01:44,320 --> 00:01:41,540
they have teleoperated from a surface

35
00:01:48,130 --> 00:01:44,330
vessels so you have to imagine that we

36
00:01:50,500 --> 00:01:48,140
are working at very low depths so in the

37
00:01:52,510 --> 00:01:50,510
ideas like a few thousand meters and in

38
00:01:54,430 --> 00:01:52,520
this kind of conditions you need special

39
00:01:56,740 --> 00:01:54,440
materials and we have to reach somehow

40
00:01:59,200 --> 00:01:56,750
those environments and by attaching

41
00:02:01,600 --> 00:01:59,210
these kind of manipulators on remotely

42
00:02:03,610 --> 00:02:01,610
operated vehicles a type of unmanned

43
00:02:06,700 --> 00:02:03,620
underwater vehicle we can reach these

44
00:02:08,979 --> 00:02:06,710
environments but again to have the

45
00:02:12,070 --> 00:02:08,989
capabilities to reach these environments

46
00:02:13,899 --> 00:02:12,080
we have to use very pieces of equipments

47
00:02:17,820 --> 00:02:13,909
or large vehicles with large

48
00:02:19,960 --> 00:02:17,830
manipulators that they need a lot of

49
00:02:21,710 --> 00:02:19,970
infrastructures in order to be deployed

50
00:02:24,080 --> 00:02:21,720
and in order to be up

51
00:02:26,420 --> 00:02:24,090
and the current state of the art it's

52
00:02:29,360 --> 00:02:26,430
teleoperation so everything happens from

53
00:02:31,490 --> 00:02:29,370
a surface vessel and this is how usually

54
00:02:33,530 --> 00:02:31,500
a control room on that surface vessel

55
00:02:36,230 --> 00:02:33,540
looks like so a lot of computers a lot

56
00:02:38,060 --> 00:02:36,240
of screens you want to see your

57
00:02:40,010 --> 00:02:38,070
environment where you are working and

58
00:02:41,540 --> 00:02:40,020
then you have a lot of people there so

59
00:02:45,760 --> 00:02:41,550
you will have the scientists that are

60
00:02:48,860 --> 00:02:45,770
going to say to the operators of the

61
00:02:51,380 --> 00:02:48,870
vehicle and the manipulator what data to

62
00:02:54,710 --> 00:02:51,390
collect but what samples to grab right

63
00:03:04,070 --> 00:02:54,720
so just to give you an idea how this

64
00:03:07,040 --> 00:03:04,080
works we so this is a video with this

65
00:03:11,120 --> 00:03:07,050
kind of Tarot operation application that

66
00:03:13,970 --> 00:03:11,130
that's very slow sorry for this so this

67
00:03:17,080 --> 00:03:13,980
is doing our Costa Rica field trips last

68
00:03:19,640 --> 00:03:17,090
December and the idea is to go and grab

69
00:03:21,710 --> 00:03:19,650
samples I'm sorry for the video I don't

70
00:03:24,830 --> 00:03:21,720
know what doing that but as you can see

71
00:03:27,020 --> 00:03:24,840
it's very very slow so the arm moves

72
00:03:29,270 --> 00:03:27,030
tries to pick a sample but then you have

73
00:03:31,280 --> 00:03:29,280
to orientate all the cameras and all of

74
00:03:34,010 --> 00:03:31,290
these the operators and the scientists

75
00:03:35,840 --> 00:03:34,020
have to do and this video it's actually

76
00:03:38,479 --> 00:03:35,850
five-time at the speed of the normal

77
00:03:41,150 --> 00:03:38,489
video so just for collecting that sample

78
00:03:42,920 --> 00:03:41,160
just pushing that push cord and

79
00:03:45,979 --> 00:03:42,930
collecting the sample and then bring it

80
00:03:52,300 --> 00:03:45,989
back to the tray all this has been done

81
00:03:58,460 --> 00:03:52,310
in more than five minutes so really

82
00:04:02,330 --> 00:03:58,470
quite a bit of time and that's very

83
00:04:04,850 --> 00:04:02,340
inefficient so what if we can improve

84
00:04:07,970 --> 00:04:04,860
this what can we what can we design in

85
00:04:10,759 --> 00:04:07,980
order to develop more autonomy for this

86
00:04:13,400 --> 00:04:10,769
kind of systems and for we want first of

87
00:04:14,900 --> 00:04:13,410
all to ease the ROV operation words like

88
00:04:17,060 --> 00:04:14,910
the operators that are doing the

89
00:04:19,430 --> 00:04:17,070
manipulation tasks and controlling the

90
00:04:21,229 --> 00:04:19,440
vehicle we want to improve the sampling

91
00:04:23,390 --> 00:04:21,239
procedure and we want to have like

92
00:04:25,220 --> 00:04:23,400
better accuracy in the area that we are

93
00:04:27,950 --> 00:04:25,230
doing the sampling we want to decrease

94
00:04:31,790 --> 00:04:27,960
the time than for this kind of sampling

95
00:04:34,159 --> 00:04:31,800
but overall what we want to achieve is

96
00:04:35,210 --> 00:04:34,169
having full autonomy so also you have to

97
00:04:37,610 --> 00:04:35,220
think that and

98
00:04:40,130 --> 00:04:37,620
inflation in general it's like done in

99
00:04:42,650 --> 00:04:40,140
factories and there everything is fully

100
00:04:44,330 --> 00:04:42,660
autonomous why is that because in those

101
00:04:46,250 --> 00:04:44,340
kind of conditions everything is

102
00:04:48,200 --> 00:04:46,260
controlled like there is like no change

103
00:04:50,330 --> 00:04:48,210
in the environment while working in

104
00:04:53,750 --> 00:04:50,340
underwater environments there's a lot of

105
00:04:55,760 --> 00:04:53,760
change everything is dynamic so in order

106
00:04:57,530 --> 00:04:55,770
for autonomy to happen we need to

107
00:05:00,170 --> 00:04:57,540
compensate for all of that and that's

108
00:05:02,030 --> 00:05:00,180
why work a lot of work is still out

109
00:05:05,690 --> 00:05:02,040
there and the current state of the art

110
00:05:07,460 --> 00:05:05,700
it's fully tell operations of art so in

111
00:05:11,120 --> 00:05:07,470
order to achieve that we started with

112
00:05:14,920 --> 00:05:11,130
the robotic system so we work at with

113
00:05:18,410 --> 00:05:14,930
Hole Oceanographic and we took the fake

114
00:05:22,340 --> 00:05:18,420
NHD that's a hybrid tattered vehicle and

115
00:05:24,920 --> 00:05:22,350
it's like this really massive vehicle

116
00:05:27,320 --> 00:05:24,930
that is rated up to 10,000 meters that

117
00:05:29,930 --> 00:05:27,330
and then we took a hydraulic

118
00:05:33,560 --> 00:05:29,940
teleoperated manipulator generally we

119
00:05:38,600 --> 00:05:33,570
attach it here you can see it in the in

120
00:05:40,970 --> 00:05:38,610
the photo and that one it's a craft

121
00:05:44,390 --> 00:05:40,980
underwater Hydra Willy Karim that has

122
00:05:49,490 --> 00:05:44,400
the capability of lifting up to a few

123
00:05:51,770 --> 00:05:49,500
hundred kilograms of payload so but all

124
00:05:54,710 --> 00:05:51,780
the system is able to operate all

125
00:05:57,770 --> 00:05:54,720
together up to 6,000 meters so we can

126
00:06:01,070 --> 00:05:57,780
reach like very low and dangerous

127
00:06:03,440 --> 00:06:01,080
environments and then we placed a bunch

128
00:06:05,810 --> 00:06:03,450
of cameras on the system we started by

129
00:06:07,640 --> 00:06:05,820
putting a camera on the end effector of

130
00:06:10,520 --> 00:06:07,650
the manipulator so that's the image that

131
00:06:13,219 --> 00:06:10,530
you see in the left and then we placed

132
00:06:16,040 --> 00:06:13,229
another pair of cameras on the vehicle

133
00:06:18,290 --> 00:06:16,050
the cameras that you see in the right in

134
00:06:22,340 --> 00:06:18,300
order to understand better the area

135
00:06:24,320 --> 00:06:22,350
where the system works and now taking

136
00:06:26,480 --> 00:06:24,330
all the system we want to facilitate the

137
00:06:28,130 --> 00:06:26,490
autonomy so how does it works the camera

138
00:06:30,320 --> 00:06:28,140
will give us information of the

139
00:06:32,780 --> 00:06:30,330
environment so we basically can do scene

140
00:06:36,080 --> 00:06:32,790
understanding autonomously so the system

141
00:06:39,680 --> 00:06:36,090
can decide where what objects to pick or

142
00:06:41,690 --> 00:06:39,690
where the sample and then we have a

143
00:06:43,610 --> 00:06:41,700
motion and interaction planning

144
00:06:45,950 --> 00:06:43,620
algorithm that is going to decide how

145
00:06:47,870 --> 00:06:45,960
the arm should move in order to pick

146
00:06:49,670 --> 00:06:47,880
that and then the controllers that are

147
00:06:51,500 --> 00:06:49,680
actually going to control the whole

148
00:06:54,440 --> 00:06:51,510
system and basically the manipulator

149
00:06:56,870 --> 00:06:54,450
force in understanding again we want to

150
00:06:59,620 --> 00:06:56,880
have like this reconstruction estimate

151
00:07:02,330 --> 00:06:59,630
the target location and orientation and

152
00:07:04,430 --> 00:07:02,340
but we also want to ensure that we are

153
00:07:06,200 --> 00:07:04,440
tracking all the time the end effector

154
00:07:08,780 --> 00:07:06,210
movement position because using

155
00:07:10,670 --> 00:07:08,790
hydraulic systems it's not really

156
00:07:12,230 --> 00:07:10,680
necessarily exact having that

157
00:07:14,180 --> 00:07:12,240
information directly from the

158
00:07:16,400 --> 00:07:14,190
manipulator so having some camera

159
00:07:18,530 --> 00:07:16,410
feedback is important and all of this

160
00:07:20,420 --> 00:07:18,540
has been presented by Gideon my

161
00:07:22,550 --> 00:07:20,430
colleague yesterday in one of the poster

162
00:07:24,830 --> 00:07:22,560
sessions and now here you can see

163
00:07:26,930 --> 00:07:24,840
exactly one how one of these three

164
00:07:30,380 --> 00:07:26,940
constructions is looking like and this

165
00:07:33,650 --> 00:07:30,390
is on indoor reconstruction that we have

166
00:07:35,570 --> 00:07:33,660
at Woods Hole and you can see there's an

167
00:07:38,270 --> 00:07:35,580
object and it's like a kiddie pool with

168
00:07:40,310 --> 00:07:38,280
some sand and then here it's the

169
00:07:42,590 --> 00:07:40,320
reconstruction from actual field works

170
00:07:45,590 --> 00:07:42,600
that we had in Costa Rica and this is

171
00:07:50,150 --> 00:07:45,600
what the robot is perceiving when it's

172
00:07:52,280 --> 00:07:50,160
starting its autonomy process now we

173
00:07:54,320 --> 00:07:52,290
have the motion interaction planning so

174
00:07:57,280 --> 00:07:54,330
we want to create what are the best

175
00:08:00,770 --> 00:07:57,290
parts in order to go and sample those

176
00:08:02,960 --> 00:08:00,780
environments and all this is the

177
00:08:05,300 --> 00:08:02,970
information that it's using is the

178
00:08:07,880 --> 00:08:05,310
current state of the robot but also all

179
00:08:11,240 --> 00:08:07,890
the image is collected with the camera

180
00:08:12,830 --> 00:08:11,250
system and so on so we developed an

181
00:08:15,860 --> 00:08:12,840
algorithm here that's optimal

182
00:08:17,660 --> 00:08:15,870
model-based planning and the idea is

183
00:08:20,870 --> 00:08:17,670
that you are going to generate some sort

184
00:08:23,990 --> 00:08:20,880
of plans based on the dynamic states of

185
00:08:26,720 --> 00:08:24,000
the system and the environment and then

186
00:08:30,110 --> 00:08:26,730
you are going to decide these laws so

187
00:08:33,170 --> 00:08:30,120
that all the constraints are always come

188
00:08:37,370 --> 00:08:33,180
back home accepted in terms like instead

189
00:08:39,469 --> 00:08:37,380
if you don't want to move a sample you

190
00:08:41,600 --> 00:08:39,479
know area that you shouldn't have or if

191
00:08:43,370 --> 00:08:41,610
you want to avoid an object or if you

192
00:08:45,200 --> 00:08:43,380
don't want to hit the vehicle with the

193
00:08:47,270 --> 00:08:45,210
sample in your hand that's the most

194
00:08:49,640 --> 00:08:47,280
important thing and then in the case

195
00:08:51,530 --> 00:08:49,650
when this is fulfilled we are going to

196
00:08:54,230 --> 00:08:51,540
optimize so we obtain the best possible

197
00:08:56,750 --> 00:08:54,240
solution and you can find more details

198
00:08:58,760 --> 00:08:56,760
in a robotics conference about this and

199
00:09:01,500 --> 00:08:58,770
now that we have a planner the idea is

200
00:09:03,829 --> 00:09:01,510
how are we going to control the system

201
00:09:06,689 --> 00:09:03,839
so we actually want to move the robot

202
00:09:09,300 --> 00:09:06,699
autonomously to do that and this happens

203
00:09:10,920 --> 00:09:09,310
in the controller so they are using to

204
00:09:13,740 --> 00:09:10,930
control the motors and the servo walls

205
00:09:16,110 --> 00:09:13,750
but for hydraulic systems is really

206
00:09:18,480 --> 00:09:16,120
important to compensate for disturbances

207
00:09:22,470 --> 00:09:18,490
for noise and uncertainties in the

208
00:09:25,980 --> 00:09:22,480
environment and for this we have used a

209
00:09:28,740 --> 00:09:25,990
system that is based on a model based

210
00:09:31,560 --> 00:09:28,750
integral slightly controller and the

211
00:09:33,389 --> 00:09:31,570
idea here is that the system can adapt

212
00:09:35,670 --> 00:09:33,399
all the time and can take into account

213
00:09:38,040 --> 00:09:35,680
the uncertainties in the behavior of the

214
00:09:41,730 --> 00:09:38,050
system and then also these can be found

215
00:09:44,639 --> 00:09:41,740
in a different paper now some initial

216
00:09:50,150 --> 00:09:44,649
results these initial results are only

217
00:09:52,410 --> 00:09:50,160
with the arm itself without any kind of

218
00:09:56,220 --> 00:09:52,420
vehicle and so on and this is in a

219
00:09:57,030 --> 00:09:56,230
controlled environment and you will see

220
00:09:59,730 --> 00:09:57,040
here

221
00:10:03,120 --> 00:09:59,740
the idea was to have this manipulator

222
00:10:06,329 --> 00:10:03,130
with a pushcart in this skinny pool and

223
00:10:09,269 --> 00:10:06,339
do some sort of autonomous sampling and

224
00:10:15,750 --> 00:10:09,279
following a specific path I hope the

225
00:10:18,240 --> 00:10:15,760
video will work so you will see now how

226
00:10:21,329 --> 00:10:18,250
the this is also five times the speed

227
00:10:24,090 --> 00:10:21,339
it's still like it's just going down and

228
00:10:27,750 --> 00:10:24,100
up down and up trying to to sample in a

229
00:10:30,180 --> 00:10:27,760
certain pattern so those were the first

230
00:10:32,430 --> 00:10:30,190
experiments that we did and here is full

231
00:10:42,540 --> 00:10:32,440
autonomy in terms of like planning the

232
00:10:46,829 --> 00:10:42,550
the part of the system and and then a

233
00:10:49,769 --> 00:10:46,839
fume a month ago we had the first with

234
00:10:52,319 --> 00:10:49,779
the tests that were happening in a small

235
00:10:54,329 --> 00:10:52,329
controlled environment in a pool so we

236
00:10:56,189 --> 00:10:54,339
put here you can see just part of the

237
00:10:58,379 --> 00:10:56,199
system you can see the vehicle you can

238
00:11:02,040 --> 00:10:58,389
see the manipulated with one of the

239
00:11:04,639 --> 00:11:02,050
cameras and then like the trace with all

240
00:11:06,990 --> 00:11:04,649
the push cords that the robot has to

241
00:11:09,540 --> 00:11:07,000
pick and place and sample the

242
00:11:11,880 --> 00:11:09,550
environment unfortunately the video is

243
00:11:14,060 --> 00:11:11,890
not available but all this system is

244
00:11:17,240 --> 00:11:14,070
going to be deployed

245
00:11:19,700 --> 00:11:17,250
here in Greece in November 2009 so we

246
00:11:22,520 --> 00:11:19,710
hope to have this whole full system

247
00:11:25,400 --> 00:11:22,530
closely gathered together and collecting

248
00:11:29,660 --> 00:11:25,410
samples in a volcano at around 600

249
00:11:32,450 --> 00:11:29,670
meters depth and we still have a lot of

250
00:11:35,150 --> 00:11:32,460
work to do in order to achieve full

251
00:11:37,220 --> 00:11:35,160
autonomy in terms of improving the

252
00:11:40,190 --> 00:11:37,230
computational speeds for planners and

253
00:11:43,490 --> 00:11:40,200
then as well as for the perception side

254
00:11:45,200 --> 00:11:43,500
and scene understanding and there's a

255
00:11:47,150 --> 00:11:45,210
lot a lot of work still to be done

256
00:11:49,450 --> 00:11:47,160
before having full autonomy but at least

257
00:11:52,670 --> 00:11:49,460
we have components that can help the

258
00:11:55,340 --> 00:11:52,680
pilots in to remotely operate the system

259
00:11:57,300 --> 00:11:55,350
so thank you very much you have

260
00:11:58,700 --> 00:11:57,310
questions